Add feature of hipBLASLt online tuning #815

hanlin12-AMD · 2025-11-20T03:27:55Z

Purpose

Add feature of hipBLASLt online tuning

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Fardin Hoque <[email protected]> Signed-off-by: Wentao Ye <[email protected]> Co-authored-by: Wentao Ye <[email protected]>

…e and Unsupported type' (vllm-project#26532) Signed-off-by: yewentao256 <[email protected]>

…deprecate use_inductor in favor of backend, simplify custom_ops (vllm-project#26502) Signed-off-by: morrison-turnansky <[email protected]> Signed-off-by: Morrison Turnansky <[email protected]> Signed-off-by: Luka Govedič <[email protected]> Co-authored-by: Luka Govedič <[email protected]> Co-authored-by: Jiangyun Zhu <[email protected]>

Signed-off-by: Fardin Hoque <[email protected]>

…lm-project#26728) Signed-off-by: Jialin Ouyang <[email protected]>

…dels (vllm-project#26526) Signed-off-by: Max de Bayser <[email protected]> Co-authored-by: Ayush Singh <[email protected]>

…24354) Signed-off-by: Lu Fang <[email protected]>

…t#26732) Signed-off-by: mgoin <[email protected]>

…cifying compile sizes (vllm-project#26681) Signed-off-by: angelayi <[email protected]>

…NSE (vllm-project#26742) Signed-off-by: Jialin Ouyang <[email protected]>

Signed-off-by: youkaichao <[email protected]>

Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

…-project#24024) Signed-off-by: n1ck-guo <[email protected]> Signed-off-by: Heng Guo <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* add aiter rmsnorm and quant fusion kernel Signed-off-by: kliuae <[email protected]> * deprint Signed-off-by: kliuae <[email protected]> * disable aiter quant mm for compat Signed-off-by: kliuae <[email protected]> --------- Signed-off-by: kliuae <[email protected]> Co-authored-by: kliuae-amd <[email protected]>

…t#26602) Signed-off-by: Max de Bayser <[email protected]>

Signed-off-by: windsonsea <[email protected]>

…m-project#26723) Signed-off-by: Xiongfei Wei <[email protected]>

Co-authored-by: perzhang <[email protected]>

Signed-off-by: vllmellm <[email protected]>

…d. (alternative PR) (vllm-project#26718) Signed-off-by: Alexei V. Ivanov <[email protected]>

…#26758) Signed-off-by: Ryan Li <[email protected]>

…-project#26684) Signed-off-by: wangyafeng <[email protected]>

…26750) Signed-off-by: Ye (Charlotte) Qi <[email protected]>

… ` (vllm-project#20983) Signed-off-by: Max Wittig <[email protected]> Signed-off-by: Antoine Auger <[email protected]> Co-authored-by: Antoine Auger <[email protected]>

Signed-off-by: wangxiyuan <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

…6776) Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: tjtanaavllm <[email protected]>

…llm-project#26556) Signed-off-by: Chendi Xue <[email protected]>

…#26792) Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: ganyi <[email protected]>

[perf](moe): set fused_shared_expert to false

Signed-off-by: tjtanaavllm <[email protected]>

[Sync] Sync upstream and upgrade to version `0.11.1rc2.dev+ge9fce7b`

Ganyi/dsv3.2 rocm support

[Qwen] Add qwen3next script

* add aiter ops Signed-off-by: tjtanaavllm <[email protected]> * fix-format Signed-off-by: tjtanaavllm <[email protected]> * support fusion for aiter quant ops Signed-off-by: tjtanaavllm <[email protected]> --------- Signed-off-by: tjtanaavllm <[email protected]>

Update the deepseek readme

… table (#745) Signed-off-by: ganyi <[email protected]>

* update the condition to use bpreshuffle ptpc gemm Signed-off-by: tjtanaavllm <[email protected]> * remove print statement Signed-off-by: tjtanaavllm <[email protected]> * fix precommit Signed-off-by: tjtanaavllm <[email protected]> --------- Signed-off-by: tjtanaavllm <[email protected]>

Signed-off-by: tjtanaavllm <[email protected]>

Signed-off-by: kliuae <[email protected]>

github-actions · 2025-11-20T03:28:23Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

kfhfar and others added 30 commits October 13, 2025 18:22

[CI Perf]Prune Tests in kernel/mamba (vllm-project#26538)

577c72a

Signed-off-by: Fardin Hoque <[email protected]> Signed-off-by: Wentao Ye <[email protected]> Co-authored-by: Wentao Ye <[email protected]>

[Bug] Fix Assertion error DeepEP/csrc/kernels/intranode.cu:928: 'fals…

7200a21

…e and Unsupported type' (vllm-project#26532) Signed-off-by: yewentao256 <[email protected]>

Pruning kernel Core Tests (vllm-project#26727)

fa96fb9

Signed-off-by: Fardin Hoque <[email protected]>

[ResponseAPI] Further polish message serialization and unit tests (vl…

35bc22f

…lm-project#26728) Signed-off-by: Jialin Ouyang <[email protected]>

Add tests for chunked prefill and prefix cache with causal pooling mo…

d8bebb0

…dels (vllm-project#26526) Signed-off-by: Max de Bayser <[email protected]> Co-authored-by: Ayush Singh <[email protected]>

[Misc][DP] support customized aggregated logger for dp (vllm-project#…

8317f72

…24354) Signed-off-by: Lu Fang <[email protected]>

[UX] Replace VLLM_ALL2ALL_BACKEND with --all2all-backend (vllm-projec…

3e051bd

…t#26732) Signed-off-by: mgoin <[email protected]>

[compile] Enable sequence parallelism for full cuda graph without spe…

b59dd19

…cifying compile sizes (vllm-project#26681) Signed-off-by: angelayi <[email protected]>

[Easy] Fix env type check errors from VLLM_DEBUG_LOG_API_SERVER_RESPO…

cfded80

…NSE (vllm-project#26742) Signed-off-by: Jialin Ouyang <[email protected]>

[build][torch.compile] upgrade depyf version (vllm-project#26702)

8a0af6a

Signed-off-by: youkaichao <[email protected]>

[torch.compile] Unwrap fused_marlin_moe custom op (vllm-project#26739)

8ae1692

Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

[Feature][Quantization] auto_round format add support for regex (vllm…

2935092

…-project#24024) Signed-off-by: n1ck-guo <[email protected]> Signed-off-by: Heng Guo <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Add support for the /rerank endpoint in vllm bench serve (vllm-projec…

fe3edb4

…t#26602) Signed-off-by: Max de Bayser <[email protected]>

[Docs] Add a start tag to build.inc.md (vllm-project#26747)

2e36cdb

Signed-off-by: windsonsea <[email protected]>

Fix lora tests failure in TPU CI due to the removal of LoRA bias (vll…

4497c8f

…m-project#26723) Signed-off-by: Xiongfei Wei <[email protected]>

[moe](feat): fuse shared expert to moe ops (#734)

0724059

Co-authored-by: perzhang <[email protected]>

[CI] [ROCm] Automate CC list for ROCm related issue (vllm-project#26753)

4821ac1

Signed-off-by: vllmellm <[email protected]>

[ci] Adding the test-amd.yaml for test definitions for the AMD backen…

d3cc842

…d. (alternative PR) (vllm-project#26718) Signed-off-by: Alexei V. Ivanov <[email protected]>

scheduler.py: Update the name of the default scheduler. (vllm-project…

481545b

…#26758) Signed-off-by: Ryan Li <[email protected]>

[Model][Bugfix]fix ernie45 load failed due to ernie45 eplb code (vllm…

01ad27f

…-project#26684) Signed-off-by: wangyafeng <[email protected]>

[CI/Build] Use 127.0.0.1 instead of localhost in utils (vllm-project#…

d32c611

…26750) Signed-off-by: Ye (Charlotte) Qi <[email protected]>

[Bugfix][FE]: Always include usage with `--enable-force-include-usage…

fd85c9f

… ` (vllm-project#20983) Signed-off-by: Max Wittig <[email protected]> Signed-off-by: Antoine Auger <[email protected]> Co-authored-by: Antoine Auger <[email protected]>

[Plugin] Make plugin group clear (vllm-project#26757)

577d498

Signed-off-by: wangxiyuan <[email protected]>

[Bugfix] Standardize merging multimodal embeddings (vllm-project#26771)

d2f816d

Signed-off-by: DarkLight1337 <[email protected]>

[Model] Use merge_by_field_config for MM models (O-P) (vllm-project#2…

74704d4

…6776) Signed-off-by: DarkLight1337 <[email protected]>

update CODEOWNERS of dev/perf (#737)

9ed51c3

Signed-off-by: tjtanaavllm <[email protected]>

[NIXL][HeteroTP] Enable KV transfer from HND prefill to NHD decode (v…

7e6edb1

…llm-project#26556) Signed-off-by: Chendi Xue <[email protected]>

[Chore] Use max_transformers_version for Qwen-VL test (vllm-project…

d1d063a

…#26792) Signed-off-by: DarkLight1337 <[email protected]>

ganyi1996ppo and others added 27 commits October 20, 2025 12:22

resolve comments

8d436d7

Signed-off-by: ganyi <[email protected]>

fix lint

fe566f0

Signed-off-by: ganyi <[email protected]>

fix lint

3513de1

Signed-off-by: ganyi <[email protected]>

fix typo

65ddd3e

Signed-off-by: ganyi <[email protected]>

[perf](moe): set fused_shared_expert to false

3e5c4bc

Merge pull request #748 from ROCm/non_fuse

3f69ada

[perf](moe): set fused_shared_expert to false

sync upstream e9fce7b

fd25afb

Signed-off-by: tjtanaavllm <[email protected]>

upgrade version to 0.11.1rc2.dev+ge9fce7b

8202f5d

Signed-off-by: tjtanaavllm <[email protected]>

add qwen3omni30b script

22f45a0

Signed-off-by: tjtanaavllm <[email protected]>

fix known per tensor quant weight edge case

dc82390

Signed-off-by: tjtanaavllm <[email protected]>

remove direct register custom op from aiter tunedgemm

f0267e6

Signed-off-by: tjtanaavllm <[email protected]>

[Sync] Sync upstream and upgrade to version 0.11.1rc2.dev+ge9fce7b

c0a63a8

[Sync] Sync upstream and upgrade to version `0.11.1rc2.dev+ge9fce7b`

Merge branch 'dev/perf' into ganyi/dsv3.2_rocm_support

3115fa5

Merge pull request #751 from ROCm/ganyi/dsv3.2_rocm_support

83704eb

Ganyi/dsv3.2 rocm support

fix pre-commit

b299db3

fix precommit

f1d4fad

Merge pull request #732 from ROCm/zhiwei/qwen3_next

5154eeb

[Qwen] Add qwen3next script

Update the deepseek readme

7b8fad4

Merge pull request #753 from ROCm/dev/perf_sync

9134135

Update the deepseek readme

[FEAT] [Perf] Support block-size > 1 for AITER MLA by remapping block…

57faa0c

… table (#745) Signed-off-by: ganyi <[email protected]>

enable swizzle hipbmm bfloat16

9d571fe

Signed-off-by: tjtanaavllm <[email protected]>

enable swizzle hipbmm for ptpc fp8; add control flag

2613fb7

Signed-off-by: tjtanaavllm <[email protected]>

fix bug

33b374e

Signed-off-by: tjtanaavllm <[email protected]>

remove hipbmm from embedding vocab, it is not compatible

5950b75

Signed-off-by: tjtanaavllm <[email protected]>

pre-transpose hipb_mm matB

e858fc9

Signed-off-by: kliuae <[email protected]>

hanlin12-AMD requested a review from gshtras as a code owner November 20, 2025 03:27

hanlin12-AMD marked this pull request as draft November 20, 2025 03:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add feature of hipBLASLt online tuning #815

Add feature of hipBLASLt online tuning #815

Uh oh!

hanlin12-AMD commented Nov 20, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Nov 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

120 participants

Add feature of hipBLASLt online tuning #815

Are you sure you want to change the base?

Add feature of hipBLASLt online tuning #815

Uh oh!

Conversation

hanlin12-AMD commented Nov 20, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

github-actions bot commented Nov 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

120 participants

hanlin12-AMD commented Nov 20, 2025 •

edited by github-actions bot

Loading